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Abstract 

Subjects were shown navigation instructions varying in length directing 
them to move in a space represented by grids on a computer screen. They 
followed the instructions by clicking on the grids in the locations specified. 

Some subjects repeated back the instructions before following them, some did 
not, and others repeated back the instructions in reduced form, including only 
the critical words. The commands in each message were presented 
simultaneously for half of the subjects and sequentially for the others. For the 
longest messages, performance was better on the initial commands and worse on 
the final commands with simultaneous than with sequential presentation. 
Instruction repetition depressed performance, but reduced repetition removed 
this disadvantage. Effects of presentation format were attributed to visual 
scanning strategies. The advantage for reduced repetition was attributable either 
to enhanced visual scanning or to reduced output interference. A follow-up 
study with auditory presentation supported the visual scanning explanation. 
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(SEE COVER SLIDE.) Our study concerns effects of presentation format 
and readback on the comprehension and execution of navigation instructions. 

This work focuses on effective communication in a concrete domain in which it 
has life-or-death consequences, namely communication between air crews and 
air traffic controllers. We chose that domain in part because of the success of our 
earlier research in this area, in which we developed an experimental paradigm 

analogous to the natural flight situation. 

In the usual version of our laboratory task, subjects hear instructions like 
those given by air traffic controllers; they repeat the instructions aloud, as pilots 
are expected to do, and then they follow the instructions, navigating in a space 
displayed on the computer. The instructions describe movements in a grid of 
four 4X4 matrices stacked one on top of another and representing a three- 
dimensional space, as illustrated on the next slide (SEE SLIDE 1). A sample 
instruction including three units is: "Turn left two squares; climb down one 
level; move forward one step." Upon hearing such instructions, the subject 
immediately repeats them, and, next, to demonstrate comprehension, uses the 
computer mouse to follow the instructions, by clicking each appropriate square 
on the grid in the order specified, as shown on the slide. 

In the experimental paradigm we have been using and in the standard 
communication situation, the messages are presented in the auditory modality. The 
sample message was shown on the slide only for ease of exposition; the subjects did 
not see the message; they only heard it. However, with new technology (such as the 
"data link" technologies being explored by NASA and the FAA) the visual modality 
could be used instead, and the modality used may influence the listeners' ability to 
comprehend, remember, and carry out the commands in the message. Hence, in a 
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previous experiment, we compared visual and auditory presentation of messages. 

We found differences between the two modalities suggesting that subjects had 
difficulty processing the longer messages in the time allotted when they were 
presented in the visual modality. The differences we found could be due in part to 
the fact that all the words were necessarily presented sequentially in the auditory 
modality but were presented simultaneously in the visual modality, so that subjects 
given visually presented messages might not have budgeted their time sufficiently 
well and might have spent too much time on the earlier words, missing the later 
words. One purpose of the present study was to test this hypothesis by comparing 
visual messages presented simultaneously and sequentially. 

Pilots are required to read back the directions given to them by an air traffic 
controller. Likewise, in the usual version of our experimental paradigm, the subjects 
repeat, or read back, the instructions given to them before following them. 

Although readback plays an important role in terms of verification, some controllers 
and pilots see the readback procedure as superfluous and time consuming. In fact, 
there have been voices within the FAA calling for removal of the readback 
requirement altogether. Thus, it is important to evaluate the effect readback has on 
performance. In our earlier experiment, we also investigated the role of repeating 
directions aloud on the execution of these directions by comparing groups in which 
readback was required to groups in which it was not. We found that the readback 
requirement led to inferior command execution especially with longer and wordier 
messages and with messages presented visually. We hypothesized that the negative 
impact of readback was due to the harmful effects of verbal output interference 
because the disruption was greatest when verbal output was largest. A second 
purpose of the present study was to test this hypothesis by comparing full readback 
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and no readback groups to a reduced readback group, in which output is reduced 
for the wordier messages. The reduced readback procedure is like that used 
naturally by pilots, who often repeat back a condensed version of what they hear. 

In our first new experiment, subjects were given messages of six different 
lengths, ranging from one to six commands, with the commands varying in their 
wordiness, namely, the number of words used for the command. The commands 
either included redundant four-word statements (for example, "climb down one 
level") or they included minimal two-word statements (for example, "down one"). 
The messages involved all three dimensions of movement. In the simultaneous 
condition, subjects saw the messages presented at the top of the computer screen 
with one command on each line, as shown on the next slide (SEE SLIDE 2). In our 
previous study, the visual messages were shown one word per line, but we showed 
all of the words in a given command on the same line in the present experiment 
because that presentation format is closer to the one used in NASA s data-link 
research. In the sequential condition, only one command was shown on the screen 
at one time, with the commands in the same locations in which they appeared in the 
simultaneous condition (so the commands "marched" down the screen). Even in the 
sequential condition, all of the words in a given command were presented at one 
time, rather than one at a time, to equate the presentation conditions in terms of 
parsing the commands. 

The duration of the visual display was matched to the duration of the 
auditory presentation in the earlier auditory condition. For example, if a given 
command took 5 seconds to present aurally, it was shown for 5 seconds in the 
sequential condition. Likewise, if the entire message took 20 seconds to present 
aurally, it was shown for 20 seconds in the simultaneous condition. Thus, the 
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simultaneous and sequential conditions were equated in their overall presentation 
length. Note that by this procedure the redundant messages were shown for 
approximately twice as long as the minimal messages because they had twice as 
many words. 

Each subject was given 72 messages arranged in 6 blocks of trials, with 2 
messages of each length in each block, one message at each level of command 
wordiness. The subjects in each presentation condition were subdivided into three 
groups. Participants in the full readback group were told to repeat aloud all of the 
directions before following them; those in the no readback group were not told to 
repeat the instructions; and participants in the reduced readback group were told 
that they should always repeat back minimal messages even when they saw 
redundant messages. For example, if they saw "climb up one level," they were to 
repeat back just "up one." 

We tested a total of 48 subjects in this experiment, all of whom were college 
students and native speakers of English. As summarized on the next slide (SEE 
SLIDE 3), the experimental design included two between-subjects variables— 
presentation condition (simultaneous, sequential) and readback group (full 
readback, no readback, reduced readback)~and two within-subjects variables- 
message length (1-6), and command wordiness (minimal-2 words, redundant-4 
words). The dependent measures were manual movement accuracy in following the 
directions and (for the full readback and reduced readback groups only) oral 
repetition accuracy. For these measures, the responses were scored as either 
completely correct or not correct. In the case of the oral repetition responses, only 
those words included in the minimal messages were scored even for the full 
readback group when redundant messages were given and repeated back in full. 
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The results for the manual movement responses are summarized on the 
next slide (SEE SLIDE 4) as a function of message length and readback group. As 
in our previous experiments, we found that accuracy decreased monotorucally 
with increases in message length. Note that, as previously, the largest drop in 
performance generally occurred between messages of Lengths 3 and 4 and 
Lengths 4 and 5. Subjects were most accurate when no readback was required 
and more accurate with the reduced readback than with the full readback, but 
only with message lengths including three or more commands, because of 
performance near the ceiling for the shorter message lengths. 

Performance on the readback should have been equivalent in the full 
readback and reduced readback conditions for the minimal messages because the 
reduction in readback applied only to the redundant messages. In fact, as shown 
on the next slide (SEE SLIDE 5), the difference between the full and reduced 
readback conditions was much more evident with the redundant messages than 
with the minimal messages. The advantage for the reduced readback condition 
with the redundant messages is a particularly strong result when considering 
that many (if not all) participants in the full readback condition used some 
reduced form on at least some of the trials. These trials would not have been 
marked incorrect on that account because the scoring has always been based on 
only the two keywords used in the minimal messages. It is also interesting to 
note that subjects performed better with redundant messages than with minimal 
messages when either no readback was required or when readback for the two 
types of messages was in the same reduced format, perhaps because the 
redundant messages were shown for approximately twice as long. Only when 
full readback was required was performance worse with redundant messages 
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than with minimal messages. This finding is consistent with the hypothesis that 
performance suffers from verbal output interference, which would be larger 
when more words must be uttered, as in the case of the redundant messages 
relative to the minimal messages. 

In this analysis, we found no effect of presentation format (sequential 
versus simultaneous) although, as you will see, there are effects of that variable 
in another analysis. 

We also examined the oral repetition responses for subjects in the full 
readback and reduced readback conditions. These results are summarized on 
the next slide (SEE SLIDE 6) as a function of message length and readback group. 
As with the manual movement responses, accuracy decreased monotonically 
with increases in message length, with the largest drop in performance between 
messages of Lengths 3 and 4 and Lengths 4 and 5. Again, subjects were more 
accurate with the reduced readback than with the full readback, but only with 
message lengths including three or more units, when performance was not on 
the ceiling. 

Although the reduction in readback applied only to the redundant 
messages, as shown on the next slide (SEE SLIDE 7), the difference between the 
full and reduced readback conditions was evident for both levels of wordiness 
but was greater with the redundant messages than with the minimal messages. 
In our previous experiments, we have found that oral repetition is better on the 
minimal messages than on the redundant messages, and we have attributed that 
difference to the increased verbal output interference with redundant messages. 
However, in the present experiment, the advantage for minimal messages was 
evident only in the full readback condition, which is the condition we had used 
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in the past. In the reduced readback condition, performance was actually better 
for redundant messages than for minimal messages, presumably because the 
output was equivalent for the two types of messages in that case but the 
redundant messages were shown for a longer time because of the greater number 

of words. 

As for the manual movement responses, there was no effect of 
presentation format (sequential versus simultaneous) in this analysis of oral 
repetition responses. However, we also conducted a set of serial position 
analyses for the oral repetition responses, and we did find effects of presentation 
format in those analyses. On the basis of our previous experiment that included 
a comparison of messages presented in the auditory and visual modality, we had 
predicted that there would be an interaction of serial position and presentation 
format, reflecting better performance on the initial positions and worse 
performance on the later positions for the simultaneous condition relative to the 
sequential condition. This prediction followed from our hypothesis that subjects 
poorly allocated their resources when all the commands were shown 
simultaneously in the visual modality. We did in fact obtain results consistent 
with this hypothesis. 

We conducted separate analyses of the oral repetition responses for each 
message length because each length has a different number of serial positions. 

As shown on the next slide (SEE SLIDE 8), there was the expected interaction of 
presentation format and serial position for both Message Lengths 5 and 6. 
Performance was better with the simultaneous format than with the sequential 
format for the initial serial positions but the opposite held for the last two 
positions. Also note that the serial position functions show a continuous 
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decrease across positions rather than the typical bow shape, which we attribute 
to the increasing costs of output interference as subjects repeat back each 
command in the message. 

The serial position analyses on the oral repetition responses also revealed 
an interesting interaction between readback group and serial position for 
Messages Lengths 3 to 5, as shown on the next slide (SEE SLIDE 9). There was an 
increasing advantage of reduced readback relative to full readback as serial 
position increased. In other words, there was a larger effect of serial position 
with full readback than with reduced readback. This pattern is consistent with 
the hypothesis that both the advantage for reduced readback relative to full 
readback and the declining serial position effect are due to verbal output 
interference. In addition, as expected because the output for redundant and that 
for minimal messages are equivalent in the reduced readback condition, the 
three-way interaction of readback group, serial position, and message wordiness 
was significant for Message Lengths 3 and 4, which can be seen on the next slide 
(SEE SLIDE 10). The decline in oral repetition accuracy as serial position 
increased was largest by far for full readback with the redundant messages. 

These findings provide strong confirmation for our previous account of 
the effects of modality on message comprehension and for the important role of 
verbal output interference in subjects' ability to repeat back and follow the 
messages they hear. However, there is an alternative explanation for some of our 
findings in this experiment involving the comparison of the full readback and 
reduced readback conditions. It is possible that in the reduced readback 
condition, subjects developed a scanning strategy in which they ignored the 
unimportant words in the redundant messages and focused only on the crucial 
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words that also occurred in the minimal messages. Such a strategy would 
provide for better performance in the reduced readback condition on the 
redundant messages relative to the minimal messages because subjects would 
have more time to study the crucial words. In fact, we did obtain this result for 
the oral repetition responses, and it would otherwise be difficult to explain 
because output interference should be equal for the two types of messages with 
reduced readback. This strategy, though interesting, is specific to the visual 
modality and could not be employed with the auditory modality. Thus, in our 
next experiment, we examined the effects of reduced readback with auditorily 

presented messages. 

As shown on the next slide (SEE SLIDE 11), the design of this experiment 
was just like the last one except there was no distinction between simultaneous 
and sequential presentation because simultaneous presentation is impossible 
with the auditory modality. We tested a total of 24 subjects in this experiment. 

The results for the manual movement responses are summarized on the 
next slide (SEE SLIDE 12) as a function of message length and readback group. 
Again, we found that accuracy decreased monotonically with increases in 
message length, and the largest drop in performance occurred between messages 
of Lengths 3 and 4 and Lengths 4 and 5. Unlike our findings with the visual 
modality in Experiment 1, subjects were not aided by the reduced readback in 
the present experiment with the auditory modality. In fact, the reduced 
readback condition showed numerically the poorest performance at the longest 

message lengths. 

Readback should have been equivalent in the full readback and reduced 
readback conditions for the minimal messages because the reduction in readback 
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applied only to the redundant messages. However, as shown on the next slide 
(SEE SLIDE 13), there was no advantage for the reduced readback condition even 
with the redundant messages. 

We also examined the oral repetition responses for subjects in the full 
readback and reduced readback conditions. These results are summarized on 
next slide (SEE SLIDE 14) as a function of message length and readback group. 

As with the manual movement responses, accuracy decreased monotonically 
with increases in message length, with the largest drop in performance between 
messages of Lengths 3 and 4 and Lengths 4 and 5. Again, in this experiment, 
unlike Experiment 1, subjects were numerically less accurate with reduced 
readback than with full readback at least at the longest message lengths. 

Recall that the reduction in readback applied only to the redundant 
messages, but, as shown on the next slide (SEE SLIDE 15), there was no 
advantage for the reduced readback even with the redundant messages. As 
previously, we found that performance on the oral repetition was better on the 
minimal messages than on the redundant messages, in this case for both the 
reduced and full readback conditions. As shown on the following slide (SEE 
SLIDE 16), the advantage for minimal messages was not found, however, for 
messages at the shortest and longest lengths, where performance is on the ceiling 
or floor. 

In summary, as shown on the next slide (SEE SLIDE 17), although reduced 
readback had a large facilitating effect when messages were presented visually 
both with repeating back and executing movements, it gave no advantage for 
either oral repetition or manual movement responses when messages were 
presented auditorily, even with redundant messages. Therefore, we conclude 
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that the advantage for reduced readback is entirely attributable to a visual 
scanning strategy in which subjects ignored the unimportant words in the 
redundant messages and focused only on the crucial words that also occurred in 
the minimal messages. Thus, pilots and others receiving navigation instructions 
visually might be able to improve their performance by learning optimal 
scanning strategies. Also, in such cases in which messages are presented visually 
under time pressure, we offer the caveat that if the commands are presented 
simultaneously, adequate attention may not be given to the later commands. We 
cannot extend these recommendations to the auditory modality, but for both 
modalities our findings strengthen our earlier recommendation to limit 
navigational instructions to no more than three commands. 
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Experiment 1 (Visual Presentation) Design 

Between-Subjects Variables 
presentation condition 

simultaneous 
sequential 
readback group 
full readback 
reduced readback 
no readback 

Within-Subjects Variables 
message length 
1 

2 

3 

4 

5 

6 

command wordiness 
minimal--2 words 
redundant— 4 words 

Dependent Measures 

manual movement accuracy 
oral repetition accuracy 


Proportion Correct 
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Experiment 1 (Visual Presentation) 
Manual Movement Responses 



Proportion Correct 
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Experiment 1 (Visual Presentation) 
Manual Movement Responses 



Message Wordiness 



Proportion Correct 
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Experiment 1 (Visual Presentation) 
Oral Repetition Responses 



Message Wordiness 


Proportion Correct Proportion Correct 


Experiment 1 (Visual Presentation) 
Oral Repetition Responses 
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Message Length 5 



Message Length 6 



Serial Position 
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Experiment 1 (Visual Presentation) 
Oral Repetition Responses 


Message Length 3 



l 2 J 


Serial Position 


Message Length 4 



\ 2 i * 

Serial Position 


Message Length 5 



Serial Position 


Correct 


Experiment 1 (Visual Presentation) 
Oral Repetition Responses 
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Message Length 3 



Serial Position 


Message Length 4 



Serial Position 
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Experiment 2 (Auditory Presentation) Design 

Between-Subjects Variable 
readback group 

full readback 
reduced readback 
no readback 

Within-Subjects Variables 
message length 
1 
2 

3 

4 

5 

6 

command wordiness 
minimal— 2 words 
redundant— 4 words 

Dependent Measures 

manual movement accuracy 
oral repetition accuracy 


Experiment 2 (Auditory Presentation) 
Manual Movement Responses 



1.8 


Full 

Reduced 



Proportion Correct 


27 


Experiment 2 (Auditory Presentation) 
Manual Movement Responses 



Message Wordiness 



Experiment 2 (Auditory Presentation) 
Oral Repetition Responses 



1 2 3 4 5 6 

Message Length 


Proportion Correct 
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Experiment 2 (Auditory Presentation) 
Oral Repetition Responses 



Message Wordiness 


Proportion Correct 
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Experiment 2 (Auditory Presentation) 
Oral Repetition Responses 



SUMMARY 

Reduced readback 

large facilitating effect for visual 
no advantage for auditory 

CONCLUSION 

Advantage for reduced readback 

due to visual scanning strategy 
ignore unimportant words 

RECOMMENDATION FOR VISUAL PRESENTATION 
Reduce length of utterances in readback 
CAVEAT FOR VISUAL PRESENTATION 
If simultaneous commands 

inadequate attention to later commands 
RECOMMENDATION FOR BOTH MODALITIES 

Limit navigational instructions to 3 commands 


